The use of error tags in ARTFL's Encyclopédie: Does good error identification lead to good error correction?

Many corpora which are prime candidates for automatic error correction, such as the output of OCR software, and electronic texts incorporating markup tags, include information on which portions of the text are most likely to contain errors. This paper describes how the error markup tag is being incorporated in the spell-checking of an electronic version of Diderot's Encyclopddie, and evaluates whether the presence of this tag has significantly aided in correcting the errors which it marks. Although the usefulness of error tagging may vary from project to project, even as the precise way in which the tagging is done varies, error tagging does not necessarily confer any benefit in attempting to correct a given word. It may, of course, nevertheless be useful in marking errors to be fixed manually at a later stage of processing the text. 1 The Encyclopddie 1.1 Project Overview The goal of this project is ultimately to detect and correct all errors in the electronic version of the 18th century French encyclopedia of Diderot and d'Alembert, a corpus of ca. 18 million words. This text is currently under development by the Project for American and French Research on the Treasury of the French Language (ARTFL); a project overview and limited sample of searchable text from the Encyclopddie are available at: Andreev et al. (1999) also provides a thorough summary of the goals and status of the project. The electronic text was largely transcribed from the original, although parts of it were produced by optical character recognition on scanned images. Unfortunately, whether a section of text was transcribed or produced by OCR was not recorded at the time of data capture, so that the error correction strategy cannot be made sensitive to this parameter. Judging from a small hand-checked section of the text, the error rate is fairly low; about one word in 40 contains an error. It should also be added that the version of the text with which I am working has already been subjected to some corrective measures after the initial data capture stage. For example , common and easily identifiable mistakes such as the word enfant showing up as en-sant were simply globally repaired throughout the text. (The original edition of the En-cyclop~die made use of the sharp 's', which was often confused with an 'f' during data entry-cf. Figure 1.) At present, my focus is on non-word error detection and correction, …